1. Field of the Invention
The present invention relates to an automatic camera calibration method.
2. Description of the Prior Art
The inventors of the present invention have aimed at a non-contact interface with a computer, and have examined a method of detecting human motion by image processing (see the following documents 1, 2, and 3).
Document 1: Hiroki Mori, Akira Utsumi, Jun Ohya, and Masahiko Yachida. Human tracking system using adaptive camera selection. In Proc. of RO_MAN'98, pp. 494–499, 1998.
Document 2: Hiroki Mori, Akira Utsumi, Jun Ohya, and Masahiko Yachida. Examination of method of tracking a plurality of persons using asynchronous multi-viewpoint information. IEICE Technical Report, PRMU98-178, pp. 15–22, 1999.
Document 3: Howard Yang, Akira Utsumi, and Jun Ohya. Stabilization of tracking of a plurality of persons using asynchronous multi-viewpoint image. IEICE Technical Report, PRMU99-150, pp. 1–7, 1999.
Considered as information related to human motion are various types of information. Here, several operations including extraction of a facial image, the height, the color of clothes, etc. which are required to identify each of persons, detection of the position and the movement direction which are required for tracking, and a seating operation are examined. By detecting information related to the operations, applications such as interaction in a virtual environment and a monitoring system are possible.
There have been conventionally a lot of suggestions about tracking of a person using an image. Many of the suggestions are by a single- or double-eye image (see the following documents 4, 5, 6, and 7) and have some problems. For example, they cannot cope with occlusion, and a detection area is narrow.
Document 4: D. M. Gavrila and L. S. Davis. 3-d model-based tracking of humans in action: a multi-view approach. In Proc. of CVPR'96, pp. 73–80, 1996.
Document 5: Ali Azarbayejani and Alex Pentland. Real-time self-calibrating stereo person tracking using 3-d shape estimation from blob features. In 13-th International Conference on Pattern Recognition, pp. 627–632, 1996.
Document 6: C. Wren, A. Azarbayejani, T. Darrell, and A. Pentland. P finder: Real-time tracking of the human body. In SPIE proceeding vol. 2615, pp. 89–98, 1996.
Document 7: M. Patrick Johnson, P. Maes, and T. Darrell. Evolving visual routines. In Proc. of Artificial Life IV, pp. 198–209, 1994.
In order to solve the problems, a person tracking system utilizing a multi-viewpoint image has been vigorously studied in recent years (see the following documents 8, 9, and 10). By utilizing the multi-viewpoint image, it is considered that the occurrence of occlusion is reduced, thereby making more stable detection possible.
Document 8: Jakub Segen and Sarma Pingali. A camera-based system for tracking people in real time. In Proc. of 13-th International Conference on Pattern Recognition, pp. 63–67, 1996.
Document 9: Q. Cai, A. Mitiche, and J. K. Aggarwal. Tracking human motion in an inddor environment. In Proc. of 2nd International Conference on Image Processing, pp. 215–218, 1995.
Document 10: Q. Cai, J. K. Aggarwal. Tracking human motion using multiple cameras. In Proc. of 13-th International Conference on Pattern Recognition, pp. 68–72, 1996.
In order to track human motion in a wide range by such a system, however, a lot of cameras are required in correspondence with detection areas, thereby causing many problems. For example, a lot of vision systems presuppose that cameras carry out observations at the same time for three-dimensional measurement, and the system becomes complicated by introducing a synchronous mechanism used therefor. Further, a plurality of observations simultaneously carried out increases redundancy among the observations, thereby reducing the processing efficiency of the system. Further, it is difficult to previously calibrate all a lot of cameras as the number of viewpoints (the number of cameras) increases.
It is considered that the problems become significant as the number of viewpoints to be utilized increases. In the tracking system utilizing the multi-viewpoint image, the inventors of the present invention have considered that the problems caused by enlarging the scale thereof are essential.
Therefore, the estimation of the position and posture of the camera (the calibration of the camera) in the tracking system utilizing the multi-viewpoint image will be considered. In the tracking system utilizing the multi-viewpoint image, it is important to also establish a method for maintenance and management such as a correspondence to the changes in the position and posture of the camera by failure during operation in addition to prior calibration of the camera.
Several methods have already been proposed with respect to the calibration of the camera in the tracking system utilizing the multi-viewpoint image.
In the following document 11, Saito et al. have found a fundamental matrix between two reference cameras and other cameras using an observation shared therebetween, to make it easy to construct a large-scaled three-dimensional video system.
Document 11: Hideo Saito and Takeo Kanade. Shape reconstruction in projective grid space from large number of images. In Proc. of CVPR, pp. 49–54, 1999.
Furthermore, in the following document 12, Lee et al. have proposed a method of finding, by an observation shared between a reference camera and each camera utilizing a target object moving on a plane, the relative position and posture from the reference camera.
Document 12: L. Lee, R. Romano, and G. Stein. Monitoring activities from multiple video streams: Establishing a common coordinate frame. IEEE Pattern Anal. Machine Intell., Vol. 22, No. 8, pp. 758–767, 2000.
However, the methods cannot be applied to a case where an observation is not shared between a camera to be calibrated and a reference camera.
Contrary to this, the inventors and others have proposed a method of estimating the position and posture of a camera utilizing the three-dimensional motion of a person who is being tracked (see the following document 13). In this method, however, a plurality of calibrated cameras are required to find the motion of the person.
Document 13: Hirotake Yamazoe, Akira Utsumi, Nobuji Tetsutani, and Masahiko Yachida. Automatic Camera calibration method for multiple camera based human tracking system. In Proc. of IWAIT 2001, pp. 77–82, 2001.